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ABSTRACT 

Various  aptitude  test  scores  were  correlated  with  academic  grades 
received  by  military  officers  enrolled  in  the  Navy  Management  Curriculum 
at  the  United  States  Naval  Postgraduate  School,  Monterey,  California . 
The  resulting  coefficients  were  analyzed  to  determine  if  the  particular 
tests  were  valid  predictors  of  academic  achievement.  Additionally,,  an 
analysis  of  the  data  was  made  using  regression  techniques  in  an  attempt 
to  provide  an  insight  into  possible  cut-off  scores  for  use  in  selecting 
candidates  for  the  Navy  Management  Curriculum.,  The  tests  used  in  this 
study  were  the  Navy  Officer  Classification  Battery  (OCB),  the  Graduate 
Record  Examination  (GRE),  and  the  Navy  Officer  Qualification  Test  (OQT). 

The  authors  are  grateful  to  James  E.  Raynes,  Commander,  Supply 
Corps,  U.  S.  Navy  for  the  assistance  and  encouragement  he  gave  us  during 
the  course  of  this  investigation.  We  are  indebted  to  Walter  E.  Marquardt, 
Lieutenant  Commander,  Civil  Engineer  Corps,  U.  S.  Navy  for  his  patient 
and  invaluable  assistance  in  dealing  with  the  statistical  concepts 
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CHAPTER  I 
THE  PROBLEM  AND  DEFINITIONS  OF  TERMS  USED 

This  paper  may  be  considered  as  the  third  of  a  series  dealing  with 
coefficients  of  correlation  derived  from  comparing  aptitude  tests  scores 
and  academic  performance  in  the  Navy  Management  Curriculum  at  the  United 
States  Naval  Postgraduate  School,  Monterey,  California.  Previous  papers 
in  this  area  (see  Bibliography)  were  prepared  by  Lieutenant  Commander 
Bernard  J.  Bandish  and  Commanders  David  J.  Martz  and  Thomas  E.  Rushin. 

I.  THE  PROBLEM 

Statement  of  the  Problem.  This  research  was  undertaken  ( 1 )  to 
determine  if  the  Navy  Officer  Classification  Battery  (OCB)  is  a  valid 
predictor  of  academic  success  in  the  Management  Curriculum!  (2)  to 
compare  the  correlation  coefficient  of  the  OCB  with  the  coefficient  of 
the  Graduate  Record  Examination  (GRE)  to  determine  which  is  the  better 
predictor,  all  factors  considered ;  (3)  to  learn  how  the  Navy  OCB  test 
administered  from  five  to  thirteen  years  ago  correlates  with  the  OCB 
given  at  this  time;  (4)  to  determine  how  the  Navy  Officer  Qualification 
Test  (OQT)  correlates  with  the  OCB  and  with  academic  performance;  and 
(5)  to  gain  an  insight  regarding  the  establishment  of  possible  cut=off 
scores  for  use  in  selecting  Management  Curriculum  students. 


II.  DEFINITIONS  OF  TERMS  USED 

Officer  Classification  Battery  (OCB)  is  a  Navy  edited  examination 
consisting  of  five  aptitude  tests.  In  this  study  only  two  of  the 
battery  were  involved.  They  were  the  Verbal  Reasoning  Test 
(NAVPERS  18325)  and  the  Mathematics  Test  (NAVPERS  18327) .  This  study 
dealt  with  OCB  tests  administered  at  different  times.  An  OCB  test, 
referred  to  in  this  paper  as  the  "original  OCB,1"  was  given  to  applicable 
officers  shortly  before  or  after  commissioning.  Another  OCB,  which  the 
authors  will  refer  to  as  the  "second  OCB,"  was  administered  to  the 
entire  student  body  enrolled  in  the  Management  Curriculum  by  the  faculty 
of  the  Management  Department.  The  two  digit  standard  scores  resulting 
from  each  segment  of  the  OCB  were  combined  into  a  weighted  total  score 
as  described  in  Chapter  IV.  The  total  time  allowed  for  these  tests 
was  85  minutes. 

Graduate  Record  Examination  (GRE)  is  a  nationally  accepted  test 
with  a  highly  significant  reliability  coefficient  designed  to  predict 
potential  at  the  graduate  level.  This  test  is  prepared  by  the  Educational 
Testing  Service,  Princeton,  New  Jersey.  In  this  study  only  the  Aptitude 
Test,  with  its  two  segments  Quantitative  and  Verbal,  was  involved. 
For  the  purposes  of  this  paper  the  three  digit  scaled  scores  obtained 
for  each  segment  were  divided  by  ten  to  make  the  data  compatible  with 
the  two  digit  OCB  standard  scores.  The  Aptitude  Test  total  scaled  scores 
is  the  addition  of  the  two  individual  scaled  scores.  However  in  this 
paper  a  total  weighted  score  was  derived  as  described  in  Chapter  IV  which 
is  expressed  as  a  two  digit  number.  Conversions  from  total  weighted  to 
total  scaled  scores  are  indicated  on  the  X-axis  of  figures  k$   5  and  6. 
The  total  time  allowed  for  the  test  was  about  three  hours.  The  fee  for 
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this  examination  is  $2.50  per  student. 

Officer  Qualification  Test  (OQT)  was  a  Navy  edited  and  administered 

test  which  was  last  used  over  13  years  ago.  The  test  consisted  of  four 

parts  of  which  only  the  Mathematics  and  Verbal  tests  were  involved  in 
'i 

this  study.  The  total  weighted  scores  of  this  test  were  derived  in  the 
same  manner  as  was  the  OCB  total  test  scores. 

Quality  Point  Rating  (QPR)  refers  to  a  student 's  weighted  grade 
score  computed  from  the  following  table  of  standards  established  by  the 
U.  S.  Naval  Postgraduate  School: 

Performance  Grade  Quality  Point  Number 

Excellent  A  3.0 

Good  B  2.0 

Fair  C  1  o0 

Barely  Passing  D  0.0 

Failure  X  =1.0 

Multiplying  the  term  hour  value  for  a  particular  course  by  the 
quality  point  number  earned  provides  the  quality  point  rating  for  that 
course.  Adding  the  quality  points  accumulated  for  all  courses  and 
dividing  by  the  total  number  of  term  hours  will  compute  the  Quality 
Point  Rating  (QPR). 

Academic  Success  in  the  Management  Curriculum  is  defined  as  the 
successful  completion  of  the  prescribed  curriculum  supported  by  a  final 
minimum  QPR  of  2.00  which  results  in  the  awarding  of  the  degree,  Master 
of  Science  in  Management,  to  otherwise  qualified  candidates. 

Aptitude  Tests  are  paper  and  pencil  tests  which  are  designed  to 
predict  the  ability  to  learn  in  a  specific  area. 

Validity  Coefficient  of  Correlation  is  a  single  number  indicating 
to  what  extent  two  things  are  related,  i.e.,  to  what  extent  variations 
in  the  one  go  with  variations  in  the  other.  With  the  knowledge  of  how 
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an  aptitude  test  varies  with  academic  performance,  a  single  measurement 

is  made  of  the  interdependence  between  the  applicable  variables  =  Through 

the  use  of  regression  analysis  (defined  below)  predictor  data  may  be 

generated  from  a  validity  coefficient .  In  general,  when  considering 

correlations  we  may  say  that  the  strength  of  relationship  can  be 

described  roughly  as  follows  for  various  validity  coefficients i 

less  than  .20  .  .  .  Slight;  almost  negligible  relationship 

.20-. 40  .  .  .  Low  Correlation;  definite  but  small  relationship 
.40-. 70  .  .  .  Moderate  Correlation;  substantial  relationship 
.70-. 90  .  .  .  High  Correlation;  marked  relationship 
.90-1.0  .  .  .  Very  high  correlation;  very  dependable 
relationship 

Those  who  employ  tests  in  guidance  and  selection  feel  that  a 

validity  correlation  should  be  at  least  .45  for  material  usefulness  and 

.  ,  .  1 

that  best  results  come  when  the  validity  correlation  (r)  is  above  o60  0 

Regression  Analysis  which  is  sometimes  referred  to  as  "estimation" 

or  "prediction  analysis"  is  expressing  the  relationship  between  two 

variables  as  a  mathematical  function/equation,  such  as  a  straight  line 

which  is  called  the  line  of  regression.  Once  this  is  done  it  is  possible 

to  predict  one  variable  based  upon  the  results  obtained  for  the  other 

variable.  However,  due  to  sampling  variations  a  confidence  interval 

should  be  constituted  about  the  regression  line*  The  width  of  this 

confidence  band  will  be  based  upon  the  particular  degree  of  accuracy 

desired  for  a  prediction. 


1 
J.  P.  Guilford,  Fundamental  Statistics  in  Psychology  and  Education 

(first  edition  New  York  and  London:   McGraw-Hill  Book  Company,  Inc., 

1942)  p.  219 


Reliability  Coefficient  of  Correlation  is  a  correlation  of  a  test 
with  itself  to  determine  to  what  extent  results  will  be  the  same  if  the 
test  is  retaken  either  in  the  same  form,  in  alternate  form5  or  by- 
splitting  the  same  test  into  strictly  comparable  halves „  A  reliability 
correlation  coefficient  of  at  least  .80  is  desirable  for  a  test  to  be 
considered  reliable. 

Confidence  Level  as  used  in  this  study  relating  to  validity 
coefficients  is  95$  and  is  the  probability  of  being  certain  that  the 
true  correlation  coefficient  for  the  entire  population  lies  within  a 
given  range.  The  confidence  level  (interval)  used  relative  to  the 
regression  analysis  is  68$  or  plus/minus  one  standard  deviation  from 
the  line  of  regression. 


CHAPTER  II 
ASSUMPTIONS  AND  LIMITATIONS 
I.  ASSUMPTIONS 

It  is  assumed  that  by  using  statistically  accepted  methods  for 
computing  correlation  coefficients,  the  resulting  data  will  be  meaningful 
in  determining  the  validity  or  reliability  of  the  test  being  considered „ 
However,  due  consideration  must  be  given  to  the  size  of  the  sample 
utilized.  The  formulae  used  for  computing  r  from  original  data, 
assigning  statistical  reliability  to  the  coefficients  obtained,  and  for 
regression  analysis  are  shown  in  Chapter  IV .  These  formulae  were 
obtained  from  sources  as  indicated  in  applicable  footnotes,  and  are 
assumed  by  the  authors  to  be  appropriate  for  the  purposes  of  this  study . 

Courses  indicated  in  this  study  as  being  quantitative  or  verbal 
in  nature  were  so  designated  with  the  thought  that  one  of  the  tests, 
mathematics  or  verbal,  was  a  better  predictor  of  academic  achievement 
than  the  other.  For  example,  the  authors  considered  the  mathematics 
test  to  be  a  better  predictor  than  was  the  verbal  test  of  academic 
performance  for  the  course  in  Advanced  Cost  Accounting. 

Even  though  slightly  different  criteria  were  used  in  the  study  made 
in  1962  by  Commanders  Martz  and  Rushin  regarding  assignment  of  courses 
into  quantitative  and  verbal  areas,  enough  similarity  exists  to  permit 
combining  the  two  samples  of  94  students  each  into  one  sample  of  1880 
In  this  regard  only  the  test  common  to  both  groups  for  this  size  sample, 
the  GRE,  will  be  analyzed  and  conclusions  drawn  therefrom <, 


II.  LIMITATIONS 

Due  to  time  considerations  this  study  used  course  grades  generated 
up  to  and  including  the  third  term.  There  are  certain  factors  which 
may  tend  to  reflect  higher  QPRs  during  the  fourth  term  and  as  a  result 
affect  higher  total  QPRs  for  the  complete  curriculum.  Among  these 
factors  are  ( 1 )  the  fourth  term  contains  no  required  quantitative 
courses,  which  to  date  have  resulted  in  a  mean  QPR  considerably  less  than 
that  for  the  verbal  courses,  and  (2)  study  habits  should  be  reinforced 
and  improved  upon.  A  significant  number  of  students  with  QPRs  very  close 
to  2.00  at  the  end  of  the  third  term  may  be  expected  to  achieve  a  QPR 
of  2.00  or  more.  For  example,  as  of  the  end  of  the  third  term  there  were 
seven  students  who  required  only  one  grade  of  A  in  a  three  hour  course, 
combined  with  grades  of  at  least  B  in  the  other  courses  taken^  to 
acquire  a  2.00  QPR  for  the  academic  year. 

Different  criteria  in  assigning  grades  to  performance  exist  among 
individual  instructors  and  this  may  have  had  some  effect  upon  the  data 
generated. 

The  prevailing  system  of  assigning  alphabetical  course  grades  does 
not  effectively  discriminate  between  students  whose  performance  is  not 
the  same.  This  is  because  identical  grades  may  be  assigned  to  students 
even  though  individual  performance  covers  a  relatively  wide  range.  For 
example,  there  can  be  no  distinction  made  between  B+  and  B=  performance 
since  both  grades  are  assigned  a  quality  point  number  of  2.00 

Motivation  must  be  recognized  as  a  factor  which  may  result  in  a 
student  who  scored  below  the  mean  in  the  GRE,  for  example,  achieving  a 
relatively  high  QPR.  Occurrences  such  as  this  tend  to  distort  the 
correlations. 
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CHAPTER  III 
DESCRIPTION  OF  THE  FIVE  CASES  STUDIED 

I.  GENERAL 

Table  I  shows  the  breakdown  of  ranks s   designators 9   and  segments  of 
the  military  components  comprising  the  1963  student  body  represented  in 
the  five  cases  discussed  below.  The  average  age  of  the  students  was 
34  years  with  a  range  of  from  27  to  42.  The  mean  commissioned  service  was 
11  years  with  a  range  of  from  5  to  18. 

II.  CASE  1 

A  sample  of  94,  which  included  the  entire  student  officer  body 
enrolled  in  the  Management  Curriculum.  This  group  took  the  second  OCB 
and  the  GRE  tests  previously  described  in  Chapter  I.  These  tests  were 
administered  two  and  three  days  prior  to  the  commencement  of  the  first 
term. 

III.  CASE  2 

A  sample  of  58  students,  part  of  the  above  mentioned  94,  combining 
Cases  3  and  4  as  indicated  below. 

IV.  CASE  3 

A  sample  of  12  students,  part  of  the  above  58^  who  upon  commissioning 
took  the  Navy  OQT  prior  to  Navy  use  of  the  OCBo  The  data  for  this 
sample  was  furnished  by  the  Bureau  of  Naval  Personnel  and  was  also 
obtained  as  required  from  individual  personnel  jackets  in  School  files. 
This  group  was  comprised  of  male  U.  S.  Navy  officers  only.  There  was  no 
information  available  regarding  OQT  scores  for  Marine  Corps  and  Coast 
Guard  officers. 


V.  CASE  4 

A  sample  of  46,  part  of  the  above  58  students  of  Case  2S   who  had  in 
prior  years  taken  the  original  OCB.  The  data  obtained  was  from  the  same 
sources  as  indicated  for  Case  3«  This  group  was  also  composed  of  male 
U.  S.  Navy  officers.  There  was  no  information  available  regarding  OCB 
tests  taken  in  prior  years  for  Marine  Corps  or  for  Coast  Guard  officers. 

VI.  CASE  5 

A  sample  of  188  students  comprising  both  the  1962  and  1963  classes 
of  the  Management  Curriculum.  This  Case  was  set  up  to  analyze  the 
correlation  between  GRE  scores  and  academic  performance  using  the  largest 
combined  sample  available  for  study  at  this  time. 


TABLE  I. 

RANKS,  DESIGNATORS  AND  MILITARY  COMPONENTS  OF 
OFFICER  STUDENTS  OF  THE  NAVY  MANAGEMENT  CURRICULUM 
UNITED  STATES  NAVAL  POSTGRADUATE  SCHOOL  1963  CLASS 


Rank/D  e  s  ignat  or* 


Line  (1100) 
Line  (1310) 
Line  (other) 
Supply  (3100) 
CEC  (5100) 
MSC  (2300) 
NC   (2900) 
USMC** 
USCG 

Totals 


CDR 


LCDR 


3 
6 
1 

15 
4 
1 

2 
2 


1  ! 


38 


LT 

11 

20 
I 

7 
1 
2 

2 

1 

45 


TOTALS 


18 
26 

;. 

29 
5 
3 

2 

4 
5 

94 


*Rank  shown  is  rank  held  upon  reporting  to  U.  S.  Naval  Postgraduate 
School. 

**Ranks  converted  to  Navy  equivalent. 
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CHAPTER  IV 
METHODS  AND  PROCEDURES 
I.  GENERAL 
The  basic  data  used  in  this  study  was  the  academic  scores  achieved 
and  aptitude  test  scores  previously  described  in  Chapter  L  The  academic 
and  test  scores  are  listed  in  Appendix  A  and  B  opposite  an  identification 
code  number  randomly  assigned  to  each  student  to  preserve  anonymity . 
Total  weighted  test  scores  and  weighted  individual  QPRs  were  derived  as 
explained  in  Section  II  below.  Frequency  distributions  for  math,  verbal 5 
and  total  weighted  criteria  were  drawn  for  the  GRE^  second  OCB,  and 
QPR  scores.  These  frequency  charts  with  means  and  standard  deviations 
indicated  thereon  are  presented  as  Figures  1,  2,  and  3»  Each  frequency 

distribution  has  a  smooth  curve  fitted  in  accordance  with  procedures 

1 
described  by  Schlaifer.  EAM  punched  cards  were  prepared  which  included 

all  the  data  shown  in  Appendix  B.  Through  the  use  of  a  computer  program 

this  data  generated  correlation  coefficients „  Statistical  reliability 

of  the  coefficients  was  determined  at  the  95  per  cent  level.  Through 

regression  analysis  predictor  data  was  then  obtained . 


1 
Robert  Schlaifer,  Introduction  to  Statistics  for  Business 

Decisions  (New  York;  McGraw-Hill  Book  Company,  Inc.,  1961)  pp.  108-109 
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II.  WEIGHTING 
The  test  scores  and  total  QPR  scores  were  weighted  according  to 
individual  student  term  hours  taken  in  both  quantitative  and  verbal 
courses  to  obtain  quantitative  and  verbal  course  QPR's  and  weighted  total 
test  scores.  The  data  for  student  no.  1  will  be  used  below  to  illustrate 
this  technique.  This  student  took  a  total  of  47  class  hours,  19  of  which 
were  in  the  quantitative  area  and  28  were  in  the  verbal  area.  By- 
multiplying  the  individual  course  hours  times  the  quality  points  earned 
by  virtue  of  the  grades  received  for  each  course ,  totals  of  45  and  6? 
quality  points  were  obtained  for  the  quantitative  and  verbal  areas 
respectively.  These  amounts  were  then  divided  by  the  total  class  hours 
for  quantitative  and  verbal  courses  to  obtain  a  QPR  for  each  area.  In 
this  case  45/19  =  2.37  (quantitative)  and  67/28  =  2.39  (verbal). 
Weighted  totals  for  the  GRE,  OCB  (both),  and  OQT  tests  were  derived  by 
using  as  weights  the  ratios  of  quantitative  course  hours  divided  by  total 
hours,  and  verbal  course  hours  divided  by  total  hours.  In  this  case  the 
ratios  were  19/47  and  28/47  or  .40  and  .60.  These  ratios  were  applied 
as  follows  using  the  original  OCB  to  illustrates 

Math  Standard  Score  times  quantitative  ratio   43  x  .40  =  17°2 

Verbal  Standard  Score  times  verbal  ratio      47  x  .60  =  28.2 

Total  Weighted  Original  OCB  score  45 °4 

The  above  procedure  was  used  to  obtain  weighted  total  scores  for 

all  tests  involved  in  this  study  and  to  obtain  quantitative  and  verbal 

QPR  scores  for  all  students.  These  weighted  scores  are  presented  in 

Appendices  A  and  B. 
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III.  CORRELATION  ANALYSIS 

The  Appendix  B  data  was  key  punched  onto  EAM  cards  which  were 
segregated  into  five  individual  cases.  These  cases  were  composed  of  the 
variables  shown  below; 

Case  1 .  9U   Students  -  9  variables 

GRE  Math  Score 

GRE  Verbal  Score 

GRE  Weighted  Total  Score 

Second  OCB  Math  Score 

Second  OCB  Verbal  Score 

Second  OCB  Weighted  Total  Score 

QPR  Weighted  Quantitative  Score 

QPR  Weighted  Verbal  Score 

QPR  Total  Score 

Case  2.  58  Students  -  6  variables 

Original  OCB  and  OQT  Math  Score 
Original  OCB  and  OQT  Verbal  Score 
Original  OCB  and  OQT  Weighted  Total  Score 
QPR  Weighted  Quantitative  Score 
QPR  Weighted  Verbal  Score 
QPR  Total  Score 

Case  3.   12  Students  -  6  variables 

OQT  Math  Score 

OQT  Verbal  Score 

OQT  Weighted  Total  Score 

QPR  Weighted  Quantitative  Score 

QPR  Weighted  Verbal  Score 

QPR  Total  Score 

Case  A..  46  Students  -  6  variables 

Original  OCB  Math  Score 
Original  OCB  Verbal  Score 
Original  Weighted  Total  Score 
QPR  Weighted  Quantitative  Score 
QPR  Weighted  Verbal  Score 
QPR  Total  Score 


M 


Case  5«   188  Students  -  6  variables 

1962  and  1963  GRE  Math  Scores 

1962  and  1963  GRE  Verbal  Scores 

1962  and  1963  GRE  Weighted  Total  Scores 

1962  and  1963  Weighted  QPR  Quantitative  Scores 

1962  and  1963  Weighted  QPR  Verbal  Scores 

1962  and  1963  Total  QPR  Scores 

Statistical  comparison  of  this  data  was  achieved  through  the  use  of 

the  following  formula  which  was  programmed  for  use  on  the  IBM  1401 

2 
computer* 

n£xy  -  c£x)(£y) 

(1)  r 


where: 


vfe2-(£x>!]E£Y2-(£Y)3 


r   =  correlation  between  X  and  Y 
xy 

X   =  applicable  test  scores  of  the  sample 
Y   =  applicable  QPR  scores  of  the  sample 
N   =  sample  size 

This  program  yielded  a  coefficient  of  correlation  between  every 
variable  included  in  each  case.  In  addition,  the  mean  and  standard 
deviation  (of  the  sample)  was  computed  for  each  of  the  individual 
18  variables. 

The  coefficients  are  shown  in  Tables  II  through  VI  under  the 
heading  "Correlation  Analysis."  Table  VII  indicates  the  means  and 
standard  deviations  obtained. 

This  program  was  also  utilized  to  determine  the  reliability 
coefficient  of  correlation  of  the  OCB  v/hen  comparing  the  original  OCB 
versus  the  second  OCB.  The  sample  considered  was  those  students 

2 

J.  P.  Guilford,  Fundamental  Statistics  in  Psychology  and 
Education  (first  edition  New  York  and  London?  McGraw-Hill  Book 
Company,  Inc.,  1942)  p.  219 
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TABLES  OF  VALIDITY  CORRELATION  COEFFICIENTS 
TABLE  II,  CASE  1,  n  =  94 
CORRELATION  ANALYSIS 


TEST 


QUANTITATIVE 


VERBAL 


3RD  QTR 
TOTAL 


GRE-MATH 


.3628 


.3773 


,4230 
^01 


GRE-VERBAL 


,4056 


,3650 


GRE-TOTAL 


.4784 


5826 


Statistical  Reliability  -  95$  Confidence  Level 


TEST  /  QPR 


RANGE 


GRE-MATH/QPR  QUANTITATIVE 

6RE-VEhSk/qpR  VErbAL 

GRE-TOTAL/QPR  TOTAL 


c.2093  to    .5563 
.4265  to    .7035 


A481    to    .7171 


CORRELATION  ANALYSIS 


"^"^-^QPR 
TEST     * — -^^^ 

QUANTITATIVE 

VERBAL 

3RD  QTR 
TOTAL 

2nd  OCB-MATH 

.2906 

.2925 

.3140 

2nd  OCB-VERBAI 

.2404 

.4301 

.3641 

2nd  OCB-TOTAL 

,28o4 

.4209 

.3841  " 

Statistical  Reliability  -  95$ 

Confidence  Level 

TEST  /  QPR 

RANGE 

2nd  OCB-MATH/QPR  QUANTITATIVE 

J041  to  .4771 

2nH  OCB-VERBAL/OPR  VERBAL 

.2628  to  .5974 

2nd  OCB-TOTAL/OPR  TOTAL 

„2109  to  ^7? 
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TABLE  III,  CASE  2,  n  =  58 


CORRELATION  ANALYSIS 


TEST  ^\^^ 

QUANTITATIVE 

VERBAL 

3RD  QTR 
TOTAL 

Orig  OCB  plus 
OQT-Math 

.0055 

.1234 

.0477 

Orig  OCB  plus 
OQT-Verbal 

.0893 

.2486 

01866 

Orig  OCB  plus 
OQT-Total 

.0725 

.2550 

J  696 

Statistical  Reliability  -  95$  Confidence  Level 


TEST  /  QPR 

RANGE 

Orig  OCB  plus  OQT-Math/QPR  Quant 

(-)o2505  to  o2615 

Orig  OCB  plus  OQT-Verbal/QPR  Verbal 

.1 496  to  .3476 

Orig  OCB  plus  OT-Total/QPR  Total 

1-J.015V  to  .355" 

TABLE  IV,  CASE  3,  n  =  12 


CORRELATION  ANALYSIS 


■v* 1 

^^•^QPR 

TEST  ^^^^^^ 

QUANTITATIVE 

VERBAL 

3RD  QTR 
TOTAL 

OQT-Math 

o0237 

.416$ 

.2215 

.  OQT-Verbal 

.0340 

.1008 

,0253~" 

OQT-Total 

.0798 

c2965 

.2027 

Statistical  Reliability  -  95$  Confidence  Level 

TEST  /  QPR 

RANGE 

OQT-Math/QPR  Quantitative 

(-0.5663  to  .6137 

OQT-Verbal/QPR  Verbal 

(-)o4832  tc  c6848 

OQ.T-Total/£PR  Total 

, \')-3ffl  t0  °7687 
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TABLE  V,  CASE  4,  n  =  46 


CORRELATION  ANALYSIS 


-R 


TEST 


QUANTITATIVE 


VERBAL 


3RD  QTR 
TOTAL 


Prig  OCB-Math 


.0343 


0O523 


u02?1 


Prig  OCB-Verbal 


33JF 


899 


Prig  OCB-Total 


,0252 


SEE 


W~l 


Statistical  Reliability 

-  95$ 

Confidence  Level 

TEST  /  QPR 

RANGE 

Orig  OCB-Math/QPR  Quant 

(~)„2587  to  .3263 

Orig  OCB-Verbal/QPR  Verbal 

.0840  to  ,5830 

Oris  OCB-Total/OPR  Total 

t-).1520  to  .4220 

TABLE  VI,  CASE  5,  n  =  188 


CORRELATION  ANALYSIS 


TEST   ^^-^^^ 

QUANTITATIVE 

VERBAL 

3RD  QTR 
TOTAL 

GRE-Math 

.4802 

.3889 

,4449 

GRE -Verbal 

.3988" 

.5114 

« 5024 

GRE-Total 

.4455 

.4908 

,5014 

Statistical  Reliability  -  95$  Confidence  Level 


TEST  /  QPR 


RANGE 


GRE-Math/QPR  Quantitative 


.3701  to  .5903 
.4057  to  .olrT 


GRE-Verbal/QPR  Verbal 


GRE-Total/QPR  Total 


.19A0  to  .6088 
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comprising  Case  4,  sample  size  46,  who  had  taken  both  OCBs.  The 
reliability  coefficients  obtained  are  discussed  in  Chapter  V . 

IV.  STATISTICAL  RELIABILITY 
Since  the  computed  correlation  coefficient  was  derived  from  a 
limited  sample  it  can  not  exactly  represent  the  true  coefficient  (r)  of 
the  population  of  potential  officer  students 0  However,  by  determining 
the  standard  error  about  the  sample  coefficients  we  can  say,  with 
selected  degrees  of  confidence,  how  much  from  the  true  r  of  the 
population  any  sample  of  the  sizes  used  in  this  study  would  probably 
exhibit.  By  assuming  that  the  true  r  is  equal  to  our  obtained  one,  we 

can  estimate  the  standard  error  of  the  sample  coefficients  about  this 

3 

value  by  means  of  the  following  formulas 

(2)£T  =  1-r2xy 
where ; 

O   =  standard  error  of  sample  coefficients 
r 

r   =  correlation  between  X  and  Y 
xy 

N   =  sample  size 
To  provide  limits  between  which  we  can  be  95  per  cent  certain  the 
true  r  lies  the  cumulative  unit  normal  distribution  yields  1*96  as  the 
value  by  which  Uy  is  multiplied  to  establish  the  range  .  To  illustrate 
this  procedure  the  computation  for  GRE  verbal  scores  versus  QPR  verbal 
scores  for  Case  1  is  shown  below. 


3 
Ibid.,  p.  209. 
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By  using  formula  (2)  we  have? 
Ov-   =  ^(^0/   =  .0707 

limit  interval  =  r   +  1.965^.=  +  .1385 

therefore:       .565O  +  .1385  =  .7335  upper  limit 
.5650  -  .1385  =  .4265  lower  limit 

Now  we  may  say  that  we  are  95  per  cent  confident  that  the  true  r 
of  the  population  lies  between  .i+265  and  .7035.  Ranges  have  been 
computed  for  the  validity  coefficients  of  all  test  scores  versus 
performance  at  the  95  per  cent  level  of  confidence.  /These  ranges  may  be 
found  for  each  Case  in  Tables  II  through  VI  under  the  heading 
"Statistical  Reliability." 

V.  REGRESSION  ANALYSIS 

While  the  statistical  reliability  analysis  previously  described 
establishes  validity  coefficient  of  correlation  confidence  levels  for 
particular  tests,  this  information  in  itself  cannot  be  used  to  predict 
an  individual's  academic  success.  However,  through  the  use  of  regression 
analysis  for  a  particular  test  we  can  predict  the  QPR  of  an  individual^, 
within  a  desired  confidence  limit,  based  upon  his  score  on  this  test. 
This  can  be  accomplished  by  establishing  a  line  of  regression  for  known 
test  versus  academic  results  and  expressing  this  linear  function  as  an 
algebraic  equation.  In  order  to  use  the  line  of  regression  for  predict- 
ion  purposes,  the  margin  of  error  of  prediction  or  standard  deviation , 
from  a  line  of  regression  must  be  determined .  Once  this  standard  error 
has  been  computed  we  can,  based  upon  the  confidence  level  desired, 
predict  academic  success  when  we  have  known  test  results. 
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The  line  of  regression  equation  is  obtained  by  using  the  following 
4 


formula: 

(3) 
where : 

! 

Y  =  a  predicted  QPR  based  upon  a  known  test  score 


Y  =  rpLL_\/x-Mx)+  My 


r  =  r   =  correlation  between  QFR9s  (Y)  and  test  scores  (X) 

O"  y  =  standard  error  of  the  QPR's  achieved  by  the  student 
body  sample 

Cj   x  =  standard  error  of  the  test  scores  achieved  by  the 
student  body  sample 

X  =  the  test  score  achieved  by  an  individual  for  which 
prediction  of  a  QPR  is  desired 

M„  =  mean  of  the  sample  test  scores 

M  =  mean  of  the  sample  QPR  scores 

To  illustrate  this  technique  for  the  total  QPR  and  total  GRE 

results  obtained  in  Case  1,  we  have  using  formula  (3)s 


Y  =  .5826/   .3965  \  (X-54.8245)  +  2.1700  =  .0304X  +  c50 

\  7.5871  / 
This  gives  us  a  line  of  regression  equation  for  predicting  QPR8s 


based  upon  GRE  scores.  The  standard  error  of  this  equation  is 

5 
computed  using  the  following  formulas 

(4)  <^yx=<Tyyi-r2yx 

where  % 

C5  yx  =  standard  error  or  standard  deviation  of  the  estimation 
of  QPR's  (Y)  based  upon  test  scores  (X) 

£f   y  =  standard  deviation  (error)  of  the  QFR»s  achieved  by 
the  sample 

r   =  r   =  correlation  between  QPR's  (Y)  and  test  scores  (X) 
yx    xy 


4 
Ibid., pp.  213-214. 

5 
Ibid.. p.  216. 
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Again,  using  the  total  QPR  versus  GRE  scores  as  an  example  we  can 
establish  a  confidence  interval  or  band  about  the  line  of  regression 
previously  described.  At  the  68  per  cent  confidence  level,  or  plus  or 
minus  one  standard  deviation,  we  have  using  formula  (4): 
CTyx  =  .3965  -/l-(. 5826)  =  .32 
limit  interval  for  Y  =  .0304X  +  .50  +  (1.00) (.32) 
Lines  of  regression  equations  were  determined  and  are  presented  and 
plotted  as  solid  lines  in  Figures  4  and  5  for  the  1963  and  combined 
1962/1963  classes,  respectively.  Due  to  the  significant  shift  of  the 
line  of  regression  for  the  composite  group,  a  line  of  regression  was 

plotted  for  the  1962  class,  using  data  accumulated  by  Commanders  Martz 

6 
and  Rushin,  in  order  to  compare  the  two  classes.  This  chart  is  depicted 

in  Figure  6,  The  confidence  interval  in  all  Figures  was  established  as 

one  standard  deviation  and  is  represented  by  the  dotted  lines  on  each 

chart.  Using  this  confidence  level  we  can  say  we  are  68  per  cent 

confident  an  individual's  QPR  will  fall  within  the  indicated  range  based 

upon  his  test  score.  To  put  it  another  way,  he  has  an  84  per  cent 

probability  of  achieving  a  QPR  equal  to  or  greater  than  that  QPR 

indicated  by  the  intersection  of  the  lower  confidence  limit  (minus  one 

standard  deviation)  and  his  achieved  test  score. 


David  J.  Martz  and  Thomas  E.  Rushin,  "Valid  Criteria  for  Selecting 
Postgraduate  Management  School  Candidates  on  the  Basis  of  Established 
Academic  Performance  and  Various  Aptitude  Tests"  (unpublished  Research 
Paper,  United  States  Naval  Postgraduate  School,  Monterey,  California, 
May  1962)  pp.  21-22. 
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CHAPTER  V 
RESULTS  OF  THE  STUDY 
I.  CASE  1  -  GRE  AND  SECOND  OCB 

Considering  the  total  scores  for  each  test^,  analysis  of  this 
correlation  revealed  the  GRE  versus  QPR  coefficient  is  .5826  as 
compared  with  .3841  for  the  second  OCB.  In  the  quantitative  and 
verbal  areas,  individually,  the  GRE  was  found  to  possess  a  validity 
correlation  approximately  25  per  cent  greater  than  that  of  the  second  OCB, 
Additionally,  the  statistical  reliability  computation  for  the  GRE  versus 
QPR  totals  relationship  reveals  that  we  can  be  95  per  cent  confident  that 
the  true  r  of  the  population  lies  between  .4481  and  .7171c  These  limits 
for  the  second  OCB  versus  QPR  totals  are  from  .2109  to  »5573o 

The  frequency  distribution  for  both  tests 9   as  smoothed  in 
Figures  1  and  2,  pages  12  and  13,  have  a  bell-shaped  configuration  which 
illustrates  the  normal  tendency  of  these  tests  when  applied  to  a  sample 
possessing  the  same  general  characteristics  as  the  present  student  body0 

Because  of  the  superior  validity  correlation  of  the  GRE  over  the 
second  OCB,  regression  analysis  was  only  performed  for  the  GREc  Based 
upon  the  present  (1963)  class  it  was  found s  as  indicated  in  Figure  4^ 
page  26,  that  a  total  scaled  GRE  score  of  1220  acquired  by  a  potential 
student  would  mean  that  he  has  a  probability  of  .68  of  achieving  a  total 
QPR  between  2.00  and  2.65.  The  same  student  would  have  a  85  per  cent 
chance  of  obtaining  a  QPR  equal  to  or  greater  than  2.00.  A  student 
having  the  mean  total  scaled  score  of  the  sample  (11 10) s  should  achieve 
a  QPR  of  1 .83  or  greater  at  the  same  confidence  level  ( .84) . 
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From  Figure  5,  page  27,  we  found  by  combining  the  two  classes 
(1962  and  1963)  a  total  scaled  score  of  11?0  is  necessary  to  achieve 
academic  success  at  the  84  per  cent  confidence  level.  To  determine  the 
cause  for  this  shift  in  the  line  of  regression^,  regression  analysis  was 
made  for  the  1962  class  in  Figure  6,  page  2  8.  We  now  find  a  total  GRE 
scaled  score  of  only  1030  is  required  for  a  student  to  be  84  per  cent 
confident  of  achieving  a  QPR  of  2.00  or  greater „ 

The  preceding  two  paragraphs  deal  only  with  data  generated  for  three 
terms,  and  we  may  therefore  expect  that  by  using  data  generated  for  the 
full  academic  year,  cut-off  scores  would  be  lower  to  achieve  the  same  QPR 
predictions  indicated  above. 

A  disparity  between  the  1962  and  1963  classes8  total  scaled  scores 
necessary  to  predict  academic  success,  at  the  same  confidence  levels  is 
evident.  Factors  which  we  would  expect  to  preclude  such  a  disparity  ares 
(1)  the  sample  sizes  are  the  same;  (2)  the  mean  ages  of  the  two  samples 
are  within  one  year  of  each  other;  (3)  the  mean  length  of  commissioned 
service  is  within  one  year;  (4)  the  rank/designator  composition  of  the 
two  samples  is  essentially  the  same;  (5)  the  data  used  for  both  classes 
was  compiled  as  of  the  end  of  the  third  term  and  most  importantly, 
(6)  the  mean  total  scaled  GRE  test  score  of  each  sample  varies  by  less 
than  1  per  cent;  i.e.  1962  -  1120;  1963  -  1110. 

II.  CASE  2  -  ORIGINAL  OCB  AND  OQT 

In  order  to  determine  if  it  was  valid  to  combine  these  two  tests 
and  treat  them  as  one  sample  of  58,  an  inter correlation  of  the  12 
students  who  had  taken  both  the  second  OCB  and  the  OQT  was  obtained  for 
the  weighted  totals  and  produced  a  reliability  coefficient  of  only  .61 „ 
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The  difference  between  these  tests'  means  of  5.82  is  higho  Since  the 
original  OCB  is  the  same  test  as  the  second  OCB  it  is  considered  that 
Case  2  data,  which  combined  the  original  OCB  and  the  OQT^  does  not  have 
any  significant  meaning. 

III.  CASE  3  -  OQT 

The  correlation  coefficient  for  the  OQT  versus  QPR  total  scores 
is  .2027,  and  due  to  the  very  small  sample  of  12 9   produced  a  statistical 
reliability  range  from  (— ) .3533  to  .7687»  This  information  is  not 
useful  in  view  of  the  large  spread. 

IV.  CASE  U  -   ORIGINAL  OCB 

The  original  OCB  weighted  total  scores  versus  total  QPR  sccres 
yielded  the  extremely  low  validity  coefficient  of  .1350  with  a 
statistical  reliability  range  of  from  (=•)<>  1520  to  O4220„ 

The  reliability  coefficient  of  correlation  between  the  original  OCB 
and  the  second  OCB  for  the  same  students  resulted  in  the  following 
coefficients:  math  =  .6226;  verbal  =  .7404j  and  weighted  totals  =  o7348. 
These  coefficients  are  considerably  less  than  the  accepted  minimum  of  ,,80, 
V.   CASE  5  -  1962/1963  COMBINED  GRE 

This  correlation  study  produced  a  coefficient  of  .5014  for  GRE 
versus  QPR  total  scores  with  a  statistical  reliability  range  of  from 
.3940  to.6088.  It  should  be  noted  that  the  range  of  grades  for  the 
1962  class  was  from  1.54  to  3.00  QPR,  with  a  mean  total  QPR  of  2o33> 
while  the  range  for  the  1963  class  is  from  .089  to  3.00  QPR  with  a  mean 
total  QPR  of  2.17.  The  correlation  coefficient  for  the  entire  class  GRE 
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versus  QPR  totals  was  ,4839.  With  a  wider  spread  of  QPR  scores  for  the 
1963  class  the  correlation  coefficient  as  previously  noted  was  .5826. 
Regression  analysis  of  this  Case  was  discussed  in  Section  I  above . 


1 

David  J.  Martz  and  Thomas  E.  Rushin,  "Valid  Criteria  for  Selecting 

Postgraduate  Management  School  Candidates  on  the  Basis  of  Established 
Academic  Performance  and  Various  Aptitude  Tests"  (unpublished  Research 
Paper,  United  States  Naval  Postgraduate  School,  Monterey,  California, 
May  1962)  pp.  21,  22,  23  and  29. 
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CHAPTER  VI 
CONCLUSIONS  AND  RECOMMENDATIONS 
I.  CONCLUSIONS 

1 .  Considering  the  tests  involved  in  this  study,,  the  GRE  proved  to 
have  high  validity  and  is  the  best  predictor  of  academic  success  in  both 
the  quantitative  and  verbal  areas . 

2.  Neither  the  original  nor  the  second  OCB  can,  from  a  statistical 
standpoint,  be  considered  a  valid  predictor  of  academic  achievement  in 
the  curriculum. 

3.  The  OCB,  based  upon  our  sample  size  of  46,  does  not  indicate  a 
high  enough  degree  of  reliability  over  a  time  span  of  several  years. 

4°  The  OQT,  as  a  result  of  the  data  generated  from  a  sample  size 
of  12,  was  found  not  to  be  interchangeable  with  the  OCB,  and  to  have  an 
extremely  low  validity  coefficient  when  related  to  QPR  scores.  However s 
because  of  the  very  small  sample  size  involved,  these  results  are 
considered  to  be  meaningless. 

5.  Based  upon  the  regression  analysis,  cut-off  scores  for  the  GRE 
in  selecting  Management  Curriculum  students  are  extremely  difficult  to 
establish  at  this  time.  This  is  because;   (1)  our  regression  analysis 
is  applicable  for  prediction  as  of  the  end  of  the  third  term  only  and 
(2)  there  exists  a  wide  differential  in  academic  performance  between  the 
1962  and  1963  classes.  A  1962  student  with  a  total  scaled  score  of  1030 
had  an  85  per  cent  chance  of  achieving  a  QPR  of  at  least  2.00.  The  same 
GRE  score  in  the  1963  group,  at  the  same  confidence  level,  predicts  a 
QPR  of  only  1.72  or  greater. 
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Since  the  two  classes  are  essentially  statistically  identical  in 
all  respects,  excluding  such  unmeasureable  personal  factors  as 
motivation,  there  should  be  an  assignable  cause  for  this  disparity. 
This  cause,  the  authors  feel,  is  the  use  of  different  grading  standards 
for  the  two  classes.  An  indication  as  to  why  the  different  standards 
exist  may  be  the  approximately  50  per  cent  turn-over  in  faculty  members 
which  took  place  subsequent  to  the  conclusion  of  the  third  term  1962 
and  prior  to  the  commencement  of  the  1963  class „  Increasing  the  sample 
size  for  regression  analysis  as  in  Case  5  tends  to  smooth  out  these 
grading  standard  differences  and  makes  the  establishing  of  cut-off 
scores  somewhat  more  feasible.  Additionally,  a  truer  picture  of  the  GRE 
score  needed  for  academic  success  could  be  obtained  by  utilizing  QPR's 
for  the  entire  academic  year. 

II.  RECOMMENDATIONS 

1 .  It  is  recommended  the  GRE  be  administered  to  officers  after  they 
have  been  initially  selected  for  Navy  Management  Postgraduate  education 
based  upon  their  records.  Utilizing  cut-off  scores,  if  developed,  or  by 
simple  ranking,  the  final  selection  could  be  effected. 

2.  It  is  recommended  investigation  be  made  as  to  the  present 
utilization  of  the  OCB,  since  the  math  and  verbal  segments  may  prove  to 
be  unreliable  over  a  long  time  span.  Additionally 9   these  segments  have 
too  low  a  validity  for  use  in  predicting  performance  for  potential 
Navy  Management  Curriculum  students. 

3.  It  is  recommended  that  future  studies  in  the  area  of  validation 
of  tests  be  continued  with  the  aim  of  ( 1 )  extending  this  study  to  take 
in  the  fourth  term;  (2)  using  regression  techniques  as  indicated  in 
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Chapter  IV  to  develop  cut-off  score  guidelines  at  specified  levels  of 
confidence.  The  program  used  in  this  study  as  well  as  the  punched 
cards  and  other  pertinent  working  papers  are  being  retained  by  the 
Management  Department  for  future  reference. 
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